Papers with parallel decoding
Plan-then-Seam: Towards Efficient Table-to-Text Generation (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent work explicitly decomposes the generation process into content planning and surface generation stages, employing two autoregressive networks for them respectively. |
| Approach: | They propose a non-parallelelizable table-to-text model that produces outputs in parallel with one network. |
| Outcome: | The proposed model achieves 3.0 5.6 times speedup for inference time, reducing 50% parameters, while maintaining as least comparable performance against strong two-stage table-to-text competitors. |
Retrieve-and-Fill for Scenario-based Task-Oriented Semantic Parsing (2023.eacl-main)
Copied to clipboard
Akshat Shrivastava, Shrey Desai, Anchit Gupta, Ali Elkahky, Aleksandr Livshits, Alexander Zotov, Ahmed Aly
| Challenge: | Task-oriented semantic parsing models have achieved strong results in recent years, but they often face obstacles adapting to novel settings with distinct semantics and scarce data. |
| Approach: | They propose a scenario-based semantic parsing model which isolates coarse-grained and fine-grounded aspects of the task and solves them with off-the-shelf neural modules. |
| Outcome: | The proposed model outperforms previous approaches in high-resource, low-resourced, and multilingual settings, and is modular, differentiable, interpretable, and allows extra supervision from scenarios. |
Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting (P18-1)
Copied to clipboard
| Challenge: | Empirically, we achieve the new state-of-the-art on all metrics (including human evaluation) on the CNN/Daily Mail dataset, as well as significantly higher abstractiveness scores. |
| Approach: | They propose a sentence-level policy gradient method that bridges computation between two neural networks in a hierarchical way while maintaining language fluency. |
| Outcome: | The proposed model achieves state-of-the-art on all metrics and higher abstractiveness scores on the CNN/Daily Mail dataset and faster training convergence than previous models. |
Glancing Transformer for Non-Autoregressive Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing non-autoregressive neural machine translation methods are either inferior to Transformer or require multiple decoding passes, leading to reduced speedup. |
| Approach: | They propose a Glancing Language Model (GLM) for single-pass parallel generation models and Glancing Transformer (GLAT) with only single- pass decoding, GLAT is able to generate high-quality translation with 8-15 speedup. |
| Outcome: | The proposed model outperforms all previous non-autoregressive methods on multiple language directions and is nearly comparable to Transformer. |
Mask Tokens as Prophet: Fine-Grained Cache Eviction for Efficient dLLM Inference (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing cache eviction strategies for autoregressive language models fail to account for the role of mask tokens and specific characteristics in dLLMs. |
| Approach: | They propose a training-free cache eviction framework tailored to dLLMs that denies a fully masked sequence and allows parallel decoding at the expense of memory and computation. |
| Outcome: | The proposed framework reduces the cost of memory and cache eviction and improves efficiency by reducing allocation in intermediate layers and concentrating resources on prompt-preferring heads. |
Uncertainty-Aware Label Refinement for Sequence Labeling (2020.emnlp-main)
Copied to clipboard
| Challenge: | Conditional random fields (CRF) for label decoding have been a problem for many tasks. |
| Approach: | They propose a two-stage label decoding framework that model long-term label dependencies while being much more computationally efficient. |
| Outcome: | The proposed method outperforms the CRF-based methods and greatly accelerates the inference process. |
An Error-Guided Correction Model for Chinese Spelling Error Correction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing neural network approaches have achieved great progress on Chinese spelling correction, but there is still room for improvement. |
| Approach: | They propose an error-guided correction model that uses pre-trained BERT models to detect errors and integrate the error confusion set into the model. |
| Outcome: | The proposed model outperforms state-of-the-art models on widely used benchmarks and achieves superior performance on both quality and computation speed. |
Beyond Hard Masks: Progressive Token Evolution for Diffusion Language Models (2026.acl-long)
Copied to clipboard
Linhao Zhong, Linyu Wu, Bozhen Fang, Tianjian Feng, Chenchen Jing, Wen Wang, Jiaheng Zhang, Hao Chen, Chunhua Shen
| Challenge: | Existing Diffusion Language Models rely on hard binary masking and discrete token assignments, which hinder the revision of early decisions. |
| Approach: | They propose a diffusion-based language modeling approach that replaces hard binary masks with evolving soft token distributions. |
| Outcome: | The proposed approach outperforms existing DLMs on multiple benchmarks. |
Parallelism and Generation Order in Masked Diffusion Language Models: Limits Today, Potential Tomorrow (2026.findings-acl)
Copied to clipboard
Yangyang Zhong, Yanmei Gu, Zhengqing Zang, Xiaomeng Li, Yuqi Ding, Xibei Jia, Yuting Shen, Zhenzhong Lan, Liwang Zhu, Weiping Liu, Junlin Zhou, Haisheng Liu, Zhong Xin Yu, Pengxin Luo, Donglian Qi, Yunfeng Yan, Junbo Zhao
| Challenge: | Autoregressive (AR) language models dominate modern natural language processing due to strong likelihood-based training objectives and reliable left-to-right decoding. |
| Approach: | They characterize MDLM behavior along two dimensions: parallelism strength and generation order . authors propose a Generate-then-Edit paradigm that mitigates dependency loss . |
| Outcome: | The proposed model improves on tasks that require "backward information" the Generate-then-Edit paradigm improves parallel decoding efficiency while reducing dependency loss. |
Fast and Robust Early-Exiting Framework for Autoregressive Language Models with Synchronized Parallel Decoding (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to speed up inference latency have shown performance degradation caused by a state copying mechanism or numerous exit paths. |
| Approach: | They propose a framework that allocates adaptive computation paths for each token based on the complexity of generating the subsequent token. |
| Outcome: | The proposed framework outperforms existing frameworks on extensive generation tasks. |
Helping the Weak Makes You Strong: Simple Multi-Task Learning Improves Non-Autoregressive Translators (2022.emnlp-main)
Copied to clipboard
| Challenge: | Non-autoregressive (NAR) neural machine translation models require a conditional independence assumption on target sequences, resulting in less informative learning signals. |
| Approach: | They propose a model-agnostic multi-task learning framework to provide more informative learning signals for NAR models under conventional MLE training. |
| Outcome: | The proposed framework improves accuracy of multiple NAR baselines without additional decoding overhead. |
LaDiC: Are Diffusion Models Really Inferior to Autoregressive Counterparts for Image-to-Text Generation? (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing models for text-to-image generation have been underperforming in image-totext generation tasks. |
| Approach: | They propose a framework that uses a split BERT to create a dedicated latent space for captions and integrates a regularization module to manage varying text lengths. |
| Outcome: | The proposed framework achieves state-of-the-art performance on the MS COCO dataset with 38.2 BLEU@4 and 126.2 CIDEr . |
Parallel Iterative Edit Models for Local Sequence Transduction (D19-1)
Copied to clipboard
| Challenge: | Recent approaches to local sequence transduction are based on the popular encoder-decoder model for sequence to sequence learning. |
| Approach: | They propose a parallel iterative edit model for the problem of local sequence transduction arising in tasks like Grammatical error correction (GEC). |
| Outcome: | The proposed model is faster and more accurate than the current encoder-decoder model for local sequence transduction tasks like translation and paraphrasing. |
Jakiro: Boosting Speculative Decoding via Decoupled MoE (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods to accelerate large language model inference have a fundamental limitation: candidates at the same tree layer share identical feature representations, constraining diversity and diminishing overall effectiveness. |
| Approach: | They propose a decoupled mixture of experts (MoE) into a draft model to generate diverse tokens from distinct feature spaces. |
| Outcome: | The proposed approach achieves significant speedups over strong baselines, with notable improvements in non-greedy scenarios where token diversity is crucial. |
DOS: Dependency-Oriented Sampler for Masked Diffusion Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing decoding strategies for pre-trained MDLMs rely on token-level uncertainty criteria, while largely overlooking sequence-level information and inter-token dependencies. |
| Approach: | They propose a training-free decoding strategy that leverages inter-token dependencies to inform token updates during generation. |
| Outcome: | Empirical results show that the proposed approach consistently achieves superior performance on both code generation and mathematical reasoning tasks. |
UniCM: A Unified Consistency Model For Efficient Multimodal Generation and Understanding (2026.findings-acl)
Copied to clipboard
| Challenge: | Consistency models (CMs) have shown promise in the efficient generation of both image and text. |
| Approach: | They propose to use a discrete token for both image and text generation to achieve a unified denoising perspective. |
| Outcome: | The proposed model outperforms SD3 on GenEval and Image Reward while being 1.5 faster at long-sequence generating speed. |
PURE: Post-hoc Unlocking and REfinement for Discrete Diffusion Decoding (2026.findings-acl)
Copied to clipboard
| Challenge: | Masked diffusion language models (MDLMs) are limited by a monotonic unmasking policy, where committed tokens cannot be revised. |
| Approach: | They propose a training-free inference algorithm for two-phase decoding that unlocks unstable regions through deterministic window masking and stochastic leftward relaxation. |
| Outcome: | The proposed algorithm significantly improves accuracy on reasoning benchmarks on GSM8K. |
FocusLLM: Precise Understanding of Long Context by Dynamic Condensing (2025.acl-long)
Copied to clipboard
Zhenyu Li, Yike Zhang, Tengyu Pan, Yutao Sun, Zhichao Duan, Junjie Fang, Rong Han, Zixuan Wang, Jianyong Wang
| Challenge: | Existing context condensing methods cannot accurately understand the full context, as there is a considerable amount of information loss in the condensed process. |
| Approach: | They propose a framework to extend the fixed context length of any decoder-only LLM by distilling crucial information from long sequences. |
| Outcome: | The proposed framework extends the fixed context length of any decoder-only LLM, allowing it to focus on relevant information from very long sequences. |
Beyond Fully Random Masking: Attention-Guided Denoising and Optimization for Diffusion Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for full-attention dLLMs rely on random masking strategies that overlook intrinsic token dependencies. |
| Approach: | They propose an attention-guided denoising and optimization framework that aligns training and optimization with attention-derived dependencies. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on mathematical and coding benchmarks. |
Reinforcement Learning for Diffusion LLMs via Energy-Based Gibbs Alignment (2026.acl-long)
Copied to clipboard
| Challenge: | Diffusion Large Language Models (dLLMs) offer parallel decoding and bidirectional context modeling . aligning dLLms with reinforcement learning (RL) remains a challenge . |
| Approach: | They propose a variational framework that reformulates RL for dLLMs as a distribution matching problem. |
| Outcome: | The proposed framework reformulates RL for dLLMs as a distribution matching problem. |